Measuring method, measuring apparatus, and measuring system

ABSTRACT

A target-object-encompassing region representing a region causing any change between first image information and second image information is determined based on the first image information precluding a target object in an imageable area and the second image information including the target object in the imageable area. A two-dimensional size of the target object included in the target-object-encompassing region is determined based on the target-object-encompassing region and the height information.

TECHNICAL FIELD

The present invention relates to a measuring method, a measuringapparatus, and a measuring system.

BACKGROUND ART

When handling target objects with robots, it is necessary to accuratelyidentify the position and the size of each target object. PatentDocument 1 discloses a technology for estimating the size of an objectto be captured and reflected in its image according to a simple method.

CITATION LIST Patent Literature Document

Patent Document 1: Japanese Patent Application Publication No.2017-211691

SUMMARY OF INVENTION Technical Problem

Occasionally, a stereo camera may be used to produce a distance imagefor the purpose of measuring a target object. However, a stereo cameracan produce a distance image having a large amount of noise. Therefore,it is difficult to detect a region of a target object at a predeterminedheight.

For this reason, the present invention aims to provide a measuringmethod, a measuring apparatus, and a measuring system which can solvethe aforementioned problem.

Solution to Problem

In a first aspect of the present invention, a measuring method includes:

determining a target-object-encompassing region representing a regioncausing a change between first image information and second imageinformation based on the first image information precluding a targetobject in an imageable area and the second image information includingthe target object in the imageable area; and determining atwo-dimensional size of the target object included in thetarget-object-encompassing region based on thetarget-object-encompassing region and height information.

In a second aspect of the present invention, a measuring apparatusincludes: determining a target-object-encompassing region representing aregion causing a change between first image information and second imageinformation based on the first image information precluding a targetobject in an imageable area and the second image information includingthe target object in the imageable area; and determining atwo-dimensional size of the target object included in thetarget-object-encompassing region based on thetarget-object-encompassing region and height information.

In a third aspect of the present invention, a measuring system includes:determining a target-object-encompassing region representing a regioncausing a change between first image information and second imageinformation based on the first image information precluding a targetobject in an imageable area and the second image information includingthe target object in the imageable area; and determining atwo-dimensional size of the target object included in thetarget-object-encompassing region based on thetarget-object-encompassing region and height information.

Advantageous Effects of Invention

According to the present invention, it is possible to identify the sizeof a target object at a predetermined height.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is a schematic illustration of a measuring system according tothe first exemplary embodiment of the present invention.

FIG. 2 is a hardware configuration diagram of a control device accordingto the first exemplary embodiment of the present invention.

FIG. 3 is a functional block diagram of the control device according tothe first exemplary embodiment of the present invention.

FIG. 4 is a flowchart showing a flow of processes to be implemented bythe measuring system according to the first exemplary embodiment of thepresent invention.

FIG. 5 includes schematic figures showing patterns for detecting theposition of a circumscribed frame R according to the first exemplaryembodiment of the present invention.

FIG. 6 includes a schematic figure showing an imaged state of aconveying object according to a first pattern for positioning thecircumscribed frame R according to the first exemplary embodiment of thepresent invention.

FIG. 7 includes schematic figures showing imaged states of a conveyingobject according to a second pattern or a third pattern for positioningthe circumscribed frame R according to the first exemplary embodiment ofthe present invention.

FIG. 8 is a schematic figure showing an outline of identifying a contactposition according to the first exemplary embodiment of the presentinvention.

FIG. 9 is a schematic figure showing an example of display informationaccording to the first exemplary embodiment of the present invention.

FIG. 10 is a schematic figure showing an outline of calculating theposition of a conveying object at a predetermined height according tothe first exemplary embodiment of the present invention.

FIG. 11 is a schematic figure showing the relationship between acircumscribed frame R and its specific regions in a first patternaccording to the first exemplary embodiment of the present invention.

FIG. 12 is a first figure showing the relationship between acircumscribed frame R and its specific region in a second patternaccording to the first exemplary embodiment of the present invention.

FIG. 13 is a second figure showing the relationship between acircumscribed frame R and its specific regions in a second patternaccording to the first exemplary embodiment of the present invention.

FIG. 14 is a first figure showing the relationship between acircumscribed frame R and its specific region in a third patternaccording to the first exemplary embodiment of the present invention.

FIG. 15 is a second figure showing the relationship between acircumscribed frame R and its specific region in the third patternaccording to the first exemplary embodiment of the present invention.

FIG. 16 is a schematic configuration diagram of a control systemequipped with a control device according to the second exemplaryembodiment of the present invention.

FIG. 17 is a schematic diagram showing the placement relationship ofpacks serving as target objects subjected to measurement in a packstorage according to the second exemplary embodiment of the presentinvention.

FIG. 18 is a schematic configuration diagram of a measuring systemaccording to the third exemplary embodiment of the present invention.

FIG. 19 is a functional block diagram of a control device according tothe third exemplary embodiment of the present invention.

FIG. 20 is a flowchart showing a flow of processes implemented by thecontrol device according to the third exemplary embodiment of thepresent invention.

FIG. 21 is a schematic diagram showing an outline of processing of thecontrol device according to the third exemplary embodiment of thepresent invention. is a schematic configuration diagram of a controldevice according to the third exemplary embodiment of the presentinvention.

DESCRIPTION OF EXEMPLARY EMBODIMENTS

Hereinafter, a measuring system according to the first exemplaryembodiment of the present invention will be described with reference tothe accompanying drawings.

FIG. 1 is a schematic illustration of a measuring system according tothe present exemplary embodiment.

As shown in FIG. 1 , a measuring system 100 includes a control device 1,a sensor 2, and a conveying vehicle 3.

The sensor 2 is configured to measure the information relating to aconveying object 4. The sensor 2 transmits the measured informationrelating to the conveying object 4 to the control device 1.Specifically, the sensor 2 is an imaging device to capture an image in afield in which the conveying vehicle 3 can move around. For example, thesensor 2 is a depth camera or a stereo camera. The sensor 2 isconfigured to capture an image around a floor surface on which theconveying vehicle 3 can travel. According to the present exemplaryembodiment, the sensor 2 is configured to measure distance informationand image information of a circumscription about a downward axisextended from the vicinity of a ceiling for attaching the sensor 2 tothe floor surface. That is, the sensor 2 is configured to generate theimage information representing an image captured in the measurementrange of the sensor 2 and the distance information representing adistance toward each position in the measurement range of the sensor 2.For example, the distance information represents a distance from thesensor 2 with respect to each pixel in the image information of themeasurement range.

The control device 1 is configured to control the conveying vehicle 3.The control device 1 controls the conveying vehicle 3 based on theacquired information. The control device 1 communicates with the sensor2 for measuring the conveying object 4 and the conveying vehicle 3. Thecontrol device 1 is configured to acquire the image information and thedistance information from the sensor 2. The control device 1 isconfigured to identify the position at which the conveying vehicle 3comes in contact with the conveying object 4 when conveying theconveying object 4, thus controlling the conveying vehicle 3 accordingto the contact position. In this connection, the control device 1 maycontrol a single conveying vehicle 3 or a plurality of conveyingvehicles 3. The conveying vehicle 3 serves as one embodiment of robots.

FIG. 2 is a hardware configuration diagram of a control device accordingto the present exemplary embodiment.

As shown in FIG. 2 , the control device 1 is a computer server includingvarious hardware elements such as a processor 101, a ROM (Read-OnlyMemory) 102, a RAM (Random-Access Memory) 103, a storage unit 104, and acommunication module 105.

For example, the processor 101 is a CPU (Central Processing Unit), a GPU(Graphics Processing Unit), or the like. For example, the storage unit104 is a HDD (Hard-Disk Drive), a SSD (Solid-State Drive), a memorycard, or the like. In addition, the storage unit 104 may be memory suchas RAM and ROM. The communication module 105 is configured to transmitdata or receive data from external devices. For example, thecommunication module 105 communicates with external devices throughwired communication paths or wireless communication paths.

FIG. 3 is a functional block diagram of a control device according tothe present exemplary embodiment.

The control device 1 is activated upon being powered on to executecontrol programs stored in advance. Accordingly, the control device 1may demonstrate various functions such as an image-informationacquisition unit 11, a distance-information acquisition unit 12, adifference detection unit 13, a measurement unit 14, a contact-positionidentification unit 15, a conveyance control unit 16, and a display unit17.

The conveying object 4 is a target object subjected to conveyance, anexample of which is a cart or a bogie carrying loads. Under the controlof the control device 1, the conveying vehicle 3 can convey theconveying object 4. Upon receiving from the control device 1 theinformation of a contact position representing the position at which theconveying vehicle 3 comes in contact with the conveying object 4, theconveying vehicle 3 can convey the conveying object 4 by pushing ordragging it towards the contact position.

When the sensor 2 captures an image of the conveying object 4 in adirection from the upper side to the lower side, it is possible tomention two instances depending on the positional relationship betweenthe sensor 2 and the conveying object 4, e.g., an instance of thecaptured image including the side face(s) of the conveying object 4 andanother instance of the captured image precluding the side face(s) ofthe conveying object 4. When the conveying object 4 is located inproximity to the center of the measurement range of the sensor 2, forexample, the upper face of the conveying object 4 may be reflected inthe captured image but the side face(s) thereof may not be reflected inthe captured image. On the other hand, when the conveying object 4 islocated at a remote place apart from the center of the measurement rangeof the sensor 2, both the upper face and the side face(s) of theconveying object 4 may be reflected in the captured image since thesensor 2 should capture the image of the conveying object 2 from theupper side. That is, when the control device 1 detects a certain regionof the captured image reflecting the conveying object 4 by using thecaptured image, it is possible to mention two instances depending on theposition of the conveying object 4, e.g., an instance of the regionincluding the side face(s) of the conveying object 4 and anotherinstance of the region precluding the side face(s) of the conveyingobject 4. In this connection, the conveying object 4 is reflected in thecaptured image such that the upper-face region thereof will be enlargedin the captured image as the sensor 4 is positioned closer to theconveying object 4. Therefore, even when the entire region of theconveying object 4 reflected in the captured image is differentiatedaccording to the height of the conveying object 4 (i.e., the distancefrom the sensor 2) and the positional relationship between the sensor 2and the conveying object 4, the control device 1 needs to accuratelycalculate the contact position at the predetermined height at which theconveying vehicle 3 comes in contact with the conveying object 4.

FIG. 4 is a flowchart showing a flow of processes to be implemented bythe measuring system according to the present exemplary embodiment.

Next, a flow of processes to be implemented by the measuring system 100will be described in succession.

First, the sensor 2 transmits to the control device 1 a certain numberof image information such as thirty frames per second. In addition, thesensor 2 transmits to the control device 1 a certain number of distanceinformation such as thirty frames per second. In the sensor 2, thetiming to transmit the image information is assumed to match the timingto transmit the distance information. In this connection, both the imageinformation and the distance information relate to the same region ofthe captured image.

The image-information acquisition unit 11 of the control device 1acquires the image information (step S101). The distance-informationacquisition unit 12 of the control device 1 acquires the distanceinformation (step S102).

The image-information acquisition unit 11 generates a background imagebased on the image information so as to record the background image on astorage unit such as the RAM 103. Herein, generation and recording ofthe background image can be performed before detection of the conveyingobject 4. For example, the image-information acquisition unit 11 maygenerate and record the background image when the measuring system 100starts its operation or when a manager of the measuring system 100instructs recording of the background image. In this connection, thebackground image is derived from the image information corresponding tothe measurement range precluding the conveying vehicle 3, the conveyingobject 4, and other foreign matter. When the storage unit stores thebackground image, the image-information acquisition unit 11 outputs tothe difference detection unit 13 the image information received from thesensor 2. The distance-information acquisition unit 12 records thedistance information received from the sensor 2 on the storage unit suchas the RAM 103. In this connection, the image-information acquisitionunit 11 and the distance-information acquisition unit 12 may assign IDsto the image information and the distance information in order toestablish an association between the image information and the distanceinformation which are correspondingly adjusted at the transmissiontiming.

The difference detection unit 13 generates the difference informationrepresenting a difference between the background image and the imageinformation received from the image-information acquisition unit 11.Specifically, upon receiving the image information, the differencedetection unit 13 compares the image information with the backgroundimage. The difference detection unit 13 generates the differenceinformation representing a region causing any variances between theimage information and the background image (step S103). For example, theimage information and the background image may be binarized to pixelseach having “0” or “1” based on the brightness for each pixel, whereinthe difference detection unit 13 generates the difference informationrepresenting a difference for each pixel between the binarized imageinformation and the binarized background image. In the differenceinformation, a pixel having a difference represented by “1” indicatesthat some object is disposed in the measurement range. The differencedetection unit 13 outputs the difference information to the measurementunit 14.

The measurement unit 14 determines whether the acquired differenceinformation includes the conveying object 4 (step S104). For example,the measurement unit 14 determines whether the conveying vehicle 3 isdisposed in the measurement range, and then the measurement unit 14determines that the measurement range includes the conveying object 4when the difference information includes information other than theconveying vehicle 3.

The conveying vehicle 3 may detect the position information thereof soas to transmit the position information to the control device 1, or thesensor 2 may detect the position information of the conveying vehicle 3so as to transmit the position information to the control device 1. Themeasurement unit 14 determines whether the conveying vehicle 3 isdisposed in the measurement range by way of comparison between theposition information of the conveying vehicle 3 and the positioninformation of the measurement range which is measured by the sensor 2and stored in advance.

The measurement unit 14 may detect the position of the conveying vehicle3 from the image information using characteristics of the conveyingvehicle 3 (brightness, size, etc.) which are stored in advance, thusdetermining whether the conveying vehicle 3 is disposed in themeasurement range. In this connection, the methodology for themeasurement unit 14 to detect the position of the conveying vehicle 3 isnot necessarily limited to the aforementioned method.

When the conveying vehicle 3 is disposed in the measurement range, themeasurement unit 14 may generate the difference information by maskingthe region of the conveying vehicle 3 in the measurement range indicatedby the difference information.

In the above, the measurement unit 14 determines whether the conveyingvehicle 3 is disposed in the measurement range and determines whetherthe conveying object 4 is included in the measurement range when thedifference information acquired from the difference detection unit 13includes information other than the information of the conveying vehicle3; however, the methodology how to determine whether the measurementrange includes the conveying object 4 is not necessarily limited to theaforementioned method.

For example, the measurement unit 14 may determine whether thedifference information includes the conveying vehicle 4 based on theinformation representative of the prescribed size of the conveyingobject 4 even when the conveying vehicle 3 and the conveying object 4are included in the measurement range.

The measurement unit 14 determines an encompassing region forencompassing the conveying object 4. For example, the measurement unit14 determines the size of a region grouping pixels representing theexistence of a difference in the difference information. The measurementunit 14 determines a circumscribed frame R of the conveying object 4representing a circumscribed frame of a region grouping pixels eachindicating a difference “1” in the difference information when theregion has a certain size or more (step S105). The circumscribed frame Rindicates a frame of an encompassing region for encompassing the upperface and the side face(s) of the conveying object 4 serving as ameasurement target. In this connection, the methodology of determiningthe circumscribed frame R is not necessarily limited to theaforementioned method; hence, the measurement unit 14 may employ anothermethod to determine the circumscribed frame R representative of theregion of the conveying object 4 included in the difference information.

FIG. 5 includes schematic figures showing patterns for detecting theposition of the circumscribed frame R in the measurement range.

The present exemplary embodiment is designed to determine whether thepattern for detecting the position of the circumscribed frame R, whichis detected by the measurement unit 14, in the captured imagecorresponds to any one among a first pattern through a third pattern(step S106). In the following descriptions, the measurement range of thesensor 2 is partitioned by a vertical line 51 and a horizontal line 52passing through the center of the measurement range, thus realizing anupper-right partition as a first region, an upper-left partition as asecond region, a lower-left partition as a third region, and alower-right partition as a fourth region.

FIG. 5 (1) shows a first pattern for positioning the circumscribed frameR. The first pattern shows the circumscribed frame R whose four verticesare included in the first region through fourth regions, respectively.The first pattern may appear when the circumscribed frame R includingthe conveying object 4 is positioned at the center of the measurementrange.

FIG. 5 (2) shows a second pattern for positioning the circumscribedframe R. The second pattern shows the circumscribed frame R whose fourvertices are collectively included in one region among the first regionthrough the fourth region. The second pattern may appear when thecircumscribed frame R including the conveying object 4 appears solely inany one region among the first region through the fourth region.

FIG. 5 (3) and FIG. 5 (4) show a third pattern for positioning thecircumscribed frame R. The third pattern shows the circumscribed frame Rwhose four vertices are disposed over two regions. The third patternshown in FIG. 5 (3) may cover an instance in which the circumscribedframe R including the conveying object 4 extends over the first regionand the second region and another instance in which the circumscribedregion R extends over the third region and the fourth region. The thirdpattern shown in FIG. 5 (4) may cover an instance in which thecircumscribed frame R including the conveying object 4 extends over thesecond region and the third region and another instance in which thecircumscribed frame R extends over the first region and the fourthregion.

FIG. 6 includes a schematic figure showing an imaged state of aconveying object according to the first pattern for positioning thecircumscribed frame R.

When the measurement unit 14 detects the position of the circumscribedframe R according to the first pattern, the upper face of the conveyingobject 4 is reflected in the captured image but the side face(s) thereofmay not be reflected in the captured image. In addition, the upper-faceregion of the conveying object 4 will be enlarged in the captured imageas the sensor is positioned closer to the upper face of the conveyingobject 4.

FIG. 7 includes schematic figures showing imaged states of conveyingobjects according to the second pattern or the third pattern forpositioning the circumscribed frame R.

When the measurement unit 14 detects the position of the circumscribedframe R according to the second pattern, as shown in FIG. 7 (2), thecaptured image may reflect the upper face of the conveying object 4 andthe other face(s) of the conveying object 4 which can be concatenated tothe position of the sensor 2 by straight lines. When the measurementunit 14 detects the position of the circumscribed frame R according tothe third pattern, the captured image may reflect various faces of theconveying object 4 as shown in FIG. 7 (3) and FIG. 7 (4).

The measurement unit 14 determines the first pattern when coordinates ofpixels within the limit of the circumscribed frame R spread over all thefour regions, i.e., the first region through the fourth region which arepartitioned by a vertical line and a horizontal line passing through thecenter of the measurement range. The measurement unit 14 determines thesecond pattern when all the coordinates of pixels within the limit ofthe circumscribed frame R spread across a single region alone among fourregions, i.e., the first region through the fourth region which arepartitioned by a vertical line and a horizontal line passing through thecenter of the measurement range. The measurement unit 14 determines thethird pattern when coordinates of pixels within the limit of thecircumscribed frame R spread across two regions which are partitioned bya vertical line 51 passing through the center of the measurement regionor when coordinates of pixels within the limit of the circumscribedframe R spread across two regions which are partitioned by a horizontalline 52 passing through the center of the measurement range.

The measurement unit 14 determines a first specific region R1representing the upper face of the conveying object 4 in the capturedimage (step S107). Specifically, the measurement unit 14 calculates aplurality of corresponding points constituting a certain face of anobject at a predetermined height according to the relationship between aplurality of feature points, which may appear in the captured imagerelating to multiple circumscribed frames R to be identified for eachcircumscribed frame R, and the height information representing theheights of feature points of multiple circumscribed frames R as well asthe relationship between a plurality of corresponding points, which maymatch coordinates in a horizontal direction (or a horizontal position)for aligning feature points of the circumscribed frame R and which mayappear in the captured image at a predetermined height having differentcoordinates of heights, and the height information representing thepredetermined height. In this connection, a plurality of correspondingpoints constituting a certain face of an object at a predeterminedheight may include a number of estimated corresponding points which maynot appear in the captured image. The measurement unit 14 determines thefirst specific region R1 representing the upper face of the conveyingobject 4 in the captured image according to a plurality of correspondingpoints.

Based on the first specific region R1, the measurement unit 14determines a second specific region R2 representing a region of theconveying object 4 in the captured image at the height at which theconveying vehicle 3 comes in contact with the conveying object 4 (stepS108).

The contact-position identification unit 15 acquires the information ofthe second specific region R2 representing the region of the conveyingobject 4 in the captured image at a height h′ at which the conveyingvehicle 3 comes in contact with the conveying object 4. Thecontact-position identification unit 15 acquires the informationrepresentative of feature points indicated by the second specific regionR2. Based on the information of feature points indicated by the secondspecific region R2, the contact-position identification unit 15identifies the position at which the conveying vehicle 3 comes incontact with the conveying object 4 in the captured image (step S109).

FIG. 8 is a schematic figure showing an outline of identifying a contactposition.

Assuming that the second specific region R2 has a rectangular shapedefined by feature points P21, P22, P23, and P24, for example, thecontact-position identification unit 15 determines the center of any oneside of the second specific region R2 as a contact position T1 at whichthe conveying vehicle 3 comes in contact with the conveying object 4 inthe captured image. For example, the contact-position identificationunit 15 determines a first side connected between the feature points P21and P22 in a conveying direction D and a second side connected betweenthe feature points P22 and P23, wherein the contact-positionidentification unit 15 identifies the center of the second side, whichhas a normal line forming a smaller angle with the conveying direction Dwithin two normal lines applied to the first and second sides, as thecontact position T1 at which the conveying vehicle 3 comes in contactwith the conveying object 4 in the captured image. The contact-positionidentification unit 15 outputs the contact position T1 to the conveyancecontrol unit 16. For example, the conveying vehicle 3 comes in contactwith the conveying object 4 at the contact position T1 so as to draw theconveying object 4 in the conveying direction D.

The contact-position identification unit 15 may identify a center T2 ofa side of the second specific region R2, which has a normal line forminga smaller angle with the conveying direction D within adjacent sides ofthe second specific region R2 directed opposite to the conveyingdirection D, as the contact position P at which the conveying vehicle 3comes in contact with the conveying object 4 in the captured image. Thatis, the contact-position identification unit 15 determines two sideslaid oppositely to the conveying direction D, i.e., a third sideconnected between the feature points P21 and P24 and a fourth sideconnected between the feature points P24 and P23, wherein thecontact-position identification unit 15 identifies the center of thethird side having a normal line forming a smaller angle with theconveying direction D within two sides as the contact position T2 atwhich the conveying vehicle 3 comes in contact with the conveying object4 in the captured image. The contact-position identification unit 15outputs the contact position T2 to the conveyance control unit 16. Inthis case, the conveying vehicle 3 comes in contact with the conveyingobject 4 at the contact position T2 so as to travel forwards whilepushing the conveying object 4 in the conveying direction D.

The conveyance control unit 16 converts the contact positions T1, T2 inthe captured image into contact positions T1′, T2′ in the real space(step S110). For example, the conveyance control unit 16 stores inadvance the corresponding relationship between coordinates in thevirtual space indicated by the captured image and coordinates in thereal space, and therefore the conveyance control unit 16 converts thecontact positions T1, T2 into the contact positions T1′, T2′ in the realspace according to the corresponding relationship.

The conveyance control unit 16 transmits to the conveying vehicle 3 thecontact positions T1′, T2′ in the real space and the conveying directionof the conveying object 4 (step S111). The conveying vehicle 3 movestowards the contact positions T1′, T2′ and then comes in contact withthe contact positions T1′, T2′, thus conveying the conveying object 4 inthe conveying direction.

The above descriptions refer to two conveying vehicles 3 to convey theconveying object 4, whereas a plurality of conveying vehicles 3 maycommunicate with the measuring system 100 to convey the conveying object4, or a single conveying vehicle 3 may communicate with the measuringsystem 100 to convey the conveying object 4. For example, four conveyingvehicles 3 may come in contact with the conveying object 4 in four waysto convey the conveying object 4, or a single conveying vehicle 3 maycome in contact with any one face of the conveying object 4 to conveythe conveying object 4 by drawing or pushing the conveying object 4.Using an example of FIG. 8 for the sake of explanation, for example, theconveyance control unit 16 may transmit the contact position T1 to afirst conveying vehicle 3 while transmitting the contact position T3 toa second conveying vehicle 3. The first conveying vehicle 3 comes incontact with the contact position T1 of the conveying object 4. Thesecond conveying vehicle 3 comes in contact with the contact position T2of the conveying object 4. The first conveying vehicle 3 and the secondconveying vehicle 3 sandwich the conveying object 4 therebetween so asto convey the conveying object 4 in the conveying direction.

The aforementioned descriptions refer to the conveying vehicle 3 whichcomes in contact with the conveying object 4; however, this does notnecessarily limit a methodology for the conveying vehicle 3 to affectthe conveying object 4. For example, the conveying vehicle 3 may pressits instrument against the conveying object 4; the conveying vehicle 3may connect, press (or engage) its instrument with a recess or aprojection formed in the conveying object 4; the conveying vehicle 3 mayreceive a strong impact from the conveying object 4. Alternatively, theconveying vehicle 3 may hold and draw the conveying object 4 with aninstrument to hold the conveying object 4 bidirectionally.

When the first conveying vehicle 3 and the second conveying vehicle 3come in contact with the conveying object 4 to convey the conveyingobject 4, for example, the second conveying vehicle 3 may move in itsmoving direction with a first force F1 applied to the contact positionT2. The first conveying vehicle 3 may move in the conveying direction atthe same speed as the second conveying vehicle 3 with a second force F2,smaller than the first force F1, applied to the contact position T1.Thus, it is possible to convey the conveying object 4 with two vehicles,i.e., the first conveying vehicle 3 and the second conveying vehicle 3.In addition, for example, the first conveying vehicle 3 comes in contactwith the contact position T1 to draw the conveying object 4 but thesecond conveying vehicle 3 may move in the conveying direction whilecontrolling the conveying object 4 not to drift by applying force to thecontact position T2. Alternatively, the second conveying vehicle 3 maycome in contact with the contact position T2 so as to move forwardswhile pushing the conveying object 4 in the conveying direction but thefirst conveying vehicle 3 may move in the conveying direction whilecontrolling the conveying object 4 not to drift by applying force to thecontact position T1.

According to the aforementioned processing of the measuring system, itis possible to identify a contact position at which a conveying vehiclecomes in contact with a conveying object. According to theaforementioned processing of the measuring system, it is possible toidentify more accurately a contact position at which a conveying vehiclecomes in contact with a conveying object. According to theaforementioned processing of the measuring system 100, it is possible tocalculate more accurately a contact position at a prescribed height atwhich the conveying vehicle 3 can come in contact with the conveyingobject 4 irrespective of different regions to be detected with respectto the conveying object 4 reflected in the captured image according tothe height of the conveying object 4 (or a distance from the sensor 2)or the positional relationship between the conveying object 4 and thesensor 2.

FIG. 9 shows an example of display information.

The display unit 17 outputs the information identified by the controldevice 1 to a predetermined destination. For example, the display unit17 acquires the captured image subjected to processing as well as thecircumscribed frame R, the first specific region R1, and the secondspecific region R2 from the measurement unit 14. In addition, thedisplay unit 17 acquires the contact position which is calculated basedon the second specific region R2. The display unit 17 generates thedisplay information to display the circumscribed frame R, the firstspecific region R1, the second specific region R2, and the contactposition. The display unit 17 outputs the display information to thepredetermined destination. For example, the display unit 17 outputs thedisplay information to an LCD (Liquid Crystal Display), a CRT (CathodeRay Tube) display, or monitor installed in the control device 1 as wellas a terminal configured to communicate with the control device 1.Accordingly, a manager or the like can confirm the currently-controlledstate of the measuring system. In this connection, the display unit 17may generate the display information to overlay the circumscribed frameR, the first specific region R1, the second specific region R2, and thecontact position, the display information to individually display thosepieces of information, or the display information to overlay arbitrarypieces of information selected by an operator who engages in working.

The process of the measurement unit 14 to calculate first and secondspecific regions will be described in detail.

Process of Measurement Unit 14 in First Pattern

Upon determining the first pattern for positioning the circumscribedframe R of the conveying object 4, the measurement unit 14 determinesthat the circumscribed frame R corresponds to the upper face of theconveying object 4. Based on the circumscribed frame R and its height,the measurement unit 14 identifies a specific region (e.g., the first orsecond specific region) at a predetermined height of the conveyingobject 4 included in the region of the circumscribed frame R.

FIG. 10 is a schematic figure showing an outline of calculating theposition of a conveying object at a predetermined height.

FIG. 10 shows focal distances f for the sensor 2 to capture imageinformation, an arbitrary feature point P1 on the upper face of theconveying object 4 in the real space, a corresponding point P2 of theconveying object 4 in the real space with a same horizontal-directioncoordinate X as the feature point P1, a point x1 diagonal to the featurepoint P1 in the captured image, and a point x2 diagonal to thecorresponding point P2 in the captured image. In the real space, asymbol z denotes a distance of the feature point P1 measured from thesensor 2 in the height direction; a symbol h denotes a distance of thecorresponding point P2 measured from the sensor 2 in the heightdirection; a symbol X denotes a distance of the feature point P1 and thecorresponding point P2 measured from the sensor 2 in the horizontaldirection. In this case, it is possible to use two formulae, i.e.,Expression (1) and Expression (2) according to the relationship betweenthe height-direction distance and the horizontal-direction distance fromthe sensor 2 in the real space and the relationship between the focaldistance of the captured image and the horizontal-direction distance foreach point measured from the center point of the captured image.

x1/f=X/z   (1)

x2/f=X/h   (2)

The aforementioned Equation (1) and Equation (2) can be modified asfollows.

X·f=z·x1   (3)

X·f=h·x2   (4)

In the above, Expression (3) and Expression (4) can be obtained.

z·x1=h·x2   (5)

Thus, it is possible to introduce Expression (5). It is possible for thesensor 2 to obtain the point x1 diagonal to the feature point P1 on theupper face of a conveying object in its captured image as well as thedistance z and the height h in the real space. Therefore, it is possibleto calculate the point x2 in the captured image diagonal to thearbitrary corresponding point P2 in the real space according toExpression (5). In this connection, it is possible for the sensor 2 tomeasure the value of the height h at an arbitrary timing, or it ispossible to set the value of the height h as an initial value whensetting up the sensor 2; however, the methodology of acquiring the valueof the height h is not necessarily limited to specific methods. In eachexpression, the symbol “/” denotes division. In each expression, thesymbol “·” denotes multiplication.

FIG. 11 is a schematic figure showing the relationship between thecircumscribed frame R and its specific regions in the first pattern.

FIG. 11 (1) shows the conveying object 4 reflected in the captured imagewhile FIG. 11 (2) is a perspective view of the conveying object 4.Herein, a predetermined height h′ for the conveying object 4 denotes aheight at which the conveying vehicle 3 comes in contact with theconveying object 4. The height at which the conveying vehicle 3 comes incontact with the conveying object 4 is a known value according tostandards of the conveying vehicle 3. The circumscribed frame R isidentified with respect to the upper face of the conveying object 4 inthe captured image. Upon determining the first pattern for positioningthe conveying object 4, the measurement unit 14 identifies thecircumscribed frame R as the first specific region R1. In the firstpattern, the first specific region R1 can be estimated as a rendition ofthe upper face of the conveying object 4.

The measurement unit 14 acquires from the storage unit or the like thedistance information associated with the image information used toidentify the circumscribed frame R of the conveying object 4. Themeasurement unit 14 acquires the height information z (e.g., the heightinformation of the first specific region R1) with respect to the featurepoints P11, P12, P13, and P14 of the circumscribed frame R in thedistance information. The measurement unit 14 identifies the position x1and the height h′ with respect to each of the feature points P11, P12,P13, and P14 of the circumscribed frame R, and therefore the measurementunit 14 substitutes the identified position x1 for x1 of Expression (5),substitutes the identified height h′ for h of Expression (5), andsubstitutes the acquired height z for z of Expression (5), thuscalculating the point x2 in the captured image in correspondence witheach of the corresponding points P21, P22, P23, and P24 constituting theregion of the conveying object 4 at the height h′.

When the circumscribed frame R has a rectangular shape, for example, themeasurement unit 14 identifies four vertices of the rectangular shape asthe feature points P11, P12, P13, and P14 so as to calculate thecorresponding points P21, P22, P23, and

P24 at the height h′ according to Expression (5). At this time, it ispossible to obtain the height of the feature points P11, P12, P13, andP14 from the distance information for each pixel of the circumscribedframe R. In addition, the height of the corresponding points P21, P22,P23, and P24 is the prescribed height h′ at which the conveying vehicle3 comes in contact with the conveying object 4. The measurement unit 14calculates a region encompassed by the corresponding points P21, P22,P23, and P24 as the second specific region R2 representing the region ofthe conveying object 4 in the captured image at the height h′ at whichthe conveying vehicle 3 comes in contact with the conveying object 4. Inthe case of the circumscribed frame R having a shape other than arectangular shape, the measurement unit 14 may identify a plurality offeature points of the circumscribed frame R so as to calculate a regionencompassed by a plurality of corresponding points corresponding to aplurality of feature points as the second specific region R2. Themeasurement unit 14 outputs to the contact-position identification unit15 the information of the second specific region R2 representing theregion of the conveying object 4 in the captured image at the height h′at which the conveying vehicle 3 comes in contact with the conveyingobject 4.

Process of Measurement Unit 14 in Second Pattern

When the measurement unit 14 determines the second pattern forpositioning the conveying object 4, it is assumed that the circumscribedregion R may include the upper face and the side face(s) of theconveying object 4. Therefore, the measurement unit 14 identifies thefirst specific region R1 representing the upper face of the conveyingobject 4 included in the region of the circumscribed frame R accordingto the following process.

FIG. 12 is a first figure showing the relationship between thecircumscribed frame R and its specific regions in the second pattern.

Specifically, the measurement unit 14 identifies the positions offeature points in the circumscribed frame R according to thepredetermined method based on the pattern for positioning thecircumscribed frame R. Assuming the circumscribed frame R has arectangular shape in association with the second pattern for positioningthe circumscribed frame R, for example, four vertices P11, P12, P13, andP14 of the circumscribed frame R will be identified as feature points.

The measurement unit 14 acquires from a distance image and a backgroundimage or a storage unit a distance h (i.e., a distance between theheight of the feature point P13 in the real space and the height of thesensor 2) representing the height of the feature point P13 which is theclosest to the center of the captured image among the feature pointsP11, P12, P13, and P14 identified in the circumscribed frame R. Herein,the information of pixels other than the regions of the conveyingvehicle 3 and the conveying object 4 in the height direction in thedistance image and the background image indicates a distance between theheight of the floor surface and the height of the sensor 2 in thevertical direction.

The measurement unit 14 acquires from the distance image correspondingto the captured image used to identify the circumscribed frame R thedistance z representing a difference between the height of the sensor 2and the height of the feature point P11, which is the farthest from thecenter of the captured image among the feature points P11, P12, P13, andP14 identified in the circumscribed frame R. Herein, the distance zrepresents the height for each point in the upper face of the conveyingobject 4 including a real-space point corresponding to the feature pointP11. The measurement unit 14 calculates an unknown corresponding pointP23 (which is equivalent to x1 of Expression (5)) in the captured imageby setting x2 of Expression (5) as the feature point P13 identified inthe circumscribed frame R of the captured image, setting h of Expression(5) as the height of the feature point P13 in the real space, andsetting x of Expression (5) as the height of the unknown correspondingpoint P23 on the upper face conforming to the horizontal-directionposition of the feature point P13 in the real space (which is equivalentto the height of the feature point P11).

In addition, the measurement unit 14 calculates the coordinates of anunillustrated point P20 (which is equivalent to x1 of Expression (5))having the height z and the same position in the horizontal direction asthe feature point P12 in the real space by setting x2 of Expression (5)as the coordinates of the feature point P12 and setting h of Expression(5) as the height of the feature point P12 in the real space, thuscalculating the coordinates of the point P22 as an intersecting pointbetween a line segment connected between P20 and P23 and a line segmentconnected between P11 and P12. In this connection, it is possible tocalculate the point P24 as a remaining point of a rectangular shapedefined by the points P11, P22, and P23.

The measurement unit 14 identifies the corresponding points asintersecting points (P22, P24) formed between the circumscribed frameand the parallel lines which include the corresponding point P23 in thecaptured image representing an upper-face point conforming to thehorizontal-direction position of a real-space point corresponding to thefeature point P13 and which are parallel to the adjacent sides of thecircumscribed frame R defined by the feature points P12, P13, and P14(i.e., the side connected between P12 and P13 and the side connectedbetween P13, and P14). The measurement unit 14 identifies a rectangularregion defined by the corresponding points P22, P23, P24 and the featurepoint P11 representing the upper face as the first specific region R1which can be presumed as a rendition of the upper face of the conveyingobject 4 in the second pattern. The measurement unit 14 sets P11, P22,P23, and P24 as feature points (first corresponding points) of the firstspecific region R1.

FIG. 13 is a second figure showing the relationship between thecircumscribed frame R and its specific regions.

The measurement unit 14 identifies the second specific region R2 bycalculating the corresponding points P31, P32, P33, and P34 at theheight h′ at which the conveying vehicle 3 comes in contact with theconveying object 4.

The measurement unit 14 acquires from the storage unit the distanceimage corresponding to the captured image used to identify thecircumscribed frame R of the conveying object 4. The measurement unit 14acquires the height z of the feature point P11 of the firstcircumscribed region in the distance image. The measurement unit 14 setsx1 of Expression (5) as the position in the captured image correspondingto the feature point P11 identified in the first specific region R1,sets z of Expression (5) as the height of the feature point representingthe upper face of the conveying object 4, and sets h of Expression (5)as the height h′ at which the conveying vehicle 3 comes in contact withthe conveying object 4, thus inputting those values into Expression (5).Accordingly, it is possible for the measurement unit 14 to calculate theposition x2 in the captured image corresponding to the correspondingpoint P31 conforming to the horizontal-direction position of the featurepoint P11 in the real space.

Similarly, according to Expression (5), the measurement unit 14calculates the corresponding points P32, P33, P34 at the height h′ atwhich the conveying vehicle 3 comes in contact with the conveying object4 in correspondence with the feature points P22, P23, P24. Themeasurement unit 14 calculates the region defined by the correspondingpoints P31, P32, P33, and P34 (second corresponding points) as thesecond specific region R2 representing the region of the conveyingobject 4 at the height h′ at which the conveying vehicle 3 comes incontact with the conveying object 4 in the captured image. In the caseof the circumscribed frame R having a shape other than a rectangularshape, the measurement unit 14 may identify a plurality of featurepoints in the circumscribed frame R so as to calculate a region definedby a plurality of corresponding points corresponding to a plurality offeature points as the second specific region R2. The measurement unit 14outputs to the contact-position identification unit 15 the informationof the second specific region R2 representing the region of theconveying object 4 at the height h′ at which the conveying vehicle 3comes in contact with the conveying object 4 in the captured image.

First Process of Measurement Unit 14 in Third Pattern

When the measurement unit 14 determines the third pattern forpositioning the circumscribed frame R of the conveying object 4, it isassumed that the circumscribed frame R may include the upper face andthe side face(s) of the conveying object 4. Therefore, the measurementunit 14 identifies the first specific region R1 representing the upperface of the conveying object 4 included in the region of thecircumscribed frame R according to the following process.

FIG. 14 is a first figure showing the relationship between thecircumscribed frame R and its specific region in the third pattern.

Specifically, the measurement unit 14 identifies the positions offeature points in the circumscribed frame R according to thepredetermined method based on the shape of the circumscribed frame R andthe position of the circumscribed frame R. Assuming the circumscribedregion R has a rectangular shape in association with the third patternfor positioning the circumscribed frame R, for example, it is possibleto identify vertices P11, P12, P13 and P14 of the circumscribed frame Ras feature points.

The measurement unit 14 acquires from the distance image and thebackground image or the storage unit the distance h representing theheight of the feature point P13 or P14, which is the closest to thecenter of the captured image among the feature points P11, P12, P13, andP14 identified in the circumscribed frame R, (i.e., the distance betweenthe height of the sensor 2 and the height of the feature point P13 orP14 in the real space). Herein, the height-direction information ofpixels other than the regions of the conveying vehicle 3 and theconveying object 4 in the distance image and the background imagerepresents the distance between the height of the sensor 2 and theheight of the floor surface in the vertical direction.

The measurement unit 14 acquires from the distance informationcorresponding to the captured image used to identify the circumscribedframe R the distance z representing a difference between the height ofthe sensor 2 and the height of the feature point P11 or P12 which is thefarthest from the center of the captured image among the feature pointsP11, P12, P13, and P14 identified in the circumscribed frame R. Herein,the distance z is the information representing the height for each pointon the upper face of the conveying object 4 including a real-space pointcorresponding to the feature point P11 or P12. The measurement unit 14calculates an unknown corresponding point P23 in the captured image(which is equivalent to x1 of Expression (5)) by setting x2 ofExpression (5) as the feature point P13 identified in the circumscribedframe R in the captured image, setting h of Expression (5) as the heightof the feature point P13 in the real space, and setting z of Expression(5) as the height of the unknown corresponding point P23 on the upperface conforming to the horizontal-direction position of the featurepoint P13 in the real space (which is equivalent to the height of thefeature point P11). In addition, the measurement unit 14 calculates anunknown corresponding point P24 in the captured image (which isequivalent to x1 of Expression (5)) by setting x2 of Expression (5) asthe feature point P14 identified in the circumscribed frame R in thecaptured image, setting h of Expression (5) as the height of the featurepoint P14 in the real space, and setting z of Expression (5) as theheight of the unknown corresponding point P24 on the upper faceconforming to the horizontal-direction position of the feature point P14in the real space (which is equivalent to the height of the featurepoint P11).

The measurement unit 14 identifies corresponding points P23′ and P24′,at which a line segment between the corresponding points P23 and P24crosses the circumscribed frame R, as corresponding points in thecaptured image in correspondence with points on the upper face of theconveying object 4. The measurement unit 14 identifies a rectangularregion defined by the corresponding points P23′, P24′ and the featurepoints P11, P12 representing the upper face as the first specific regionR1 which can be presumed as a rendition of the upper face of theconveying object 4 in the third pattern. The measurement unit 14 setsthose points P11, P12, P23′, P24′ as the feature points (firstcorresponding points) of the first specific region R1.

The measurement unit 14 identifies the second specific region R2 bycalculating the corresponding points P31, P32, P33, P34 at the height h′at which the conveying vehicle 3 comes in contact with the conveyingobject 4.

The measurement unit 14 acquires from the storage unit the distanceimage corresponding to the captured image used to identify thecircumscribed frame R of the conveying object 4. The measurement unit 14acquires the height z of the feature point P11 of the first specificregion R1 in the distance image. The measurement unit 14 sets x1 ofExpression (5) as the position in the captured image corresponding tothe feature point P11 identified in the first specific region R1, sets zof Expression (5) as the height of the feature point P11 representingthe upper face of the conveying object 4, and sets h of Expression (5)as the height h′ at which the conveying vehicle 3 comes in contact withthe conveying object 4, thus inputting those values into Expression (5).Thus, it is possible for the measurement unit 14 to calculate theposition x2 in the captured image in correspondence with thecorresponding point P31 conforming to the horizontal-direction positionof the feature point P11 in the real space.

Similarly, according to Expression (5), the measurement unit 14calculates the corresponding points P32, P33, P34 at the height h′ atwhich the conveying vehicle 3 comes in contact with the conveying object4 in correspondence with the feature point P12, P23′, P24′. Themeasurement unit 14 calculates a region defined by the correspondingpoints P31, P32, P33, P34 (second corresponding points) as the secondspecific region R2 representing the region of the conveying object 4 inthe captured image at the height h′ at which the conveying vehicle 3comes in contact with the conveying object 4. In the case of thecircumscribed frame R having a shape other than a rectangular shape, themeasurement unit 14 may identify a plurality of feature points of thecircumscribed frame R so as to calculate a region defined by a pluralityof corresponding points corresponding to a plurality of feature pointsas the second specific region R2. The measurement unit 14 outputs to thecontact-position identification unit 15 the information of the secondspecific region R2 representing the region of the conveying object 4 inthe captured image at the height h′ at which the conveying vehicle 3comes in contact with the conveying object 4.

Second Process of Measurement Unit 14 in Third Pattern

When the measurement unit 14 determines the third pattern forpositioning the circumscribed frame R of the conveying object 4, it isassumed that the circumscribed frame R may include the upper face andthe side face(s) of the conveying object 4. Therefore, the measurementunit 14 identifies the first specific region R1 representing the upperface of the conveying object 4 included in the region indicated by thecircumscribed frame R.

FIG. 15 is a second figure showing the relationship between thecircumscribed frame R and its specific region in the third pattern.

Specifically, the measurement unit 14 identifies the positions offeature points of the circumscribed frame R according to thepredetermined method based on the shape of the circumscribed frame R andthe position of the circumscribed frame R. Assuming the circumscribedframe R has a rectangular shape in association with the third patternfor positioning the circumscribed frame R, for example, it is possibleto identify four vertices P11, P12, P13, P14 of the circumscribed frameR as its feature points.

The measurement unit 14 acquires from the distance image and thebackground image or the storage unit the distance h representing theheight of the feature point P12 or P13 which is the closest to thecenter of the captured image among the feature points P11, P12, P13, P14identified in the circumscribed frame R (i.e., the distance between theheight of the sensor 2 and the height of the feature point P12 or P13 inthe real space). Herein, the height-direction information representingpixels other than the regions of the conveying vehicle 3 and theconveying object 4 in the distance image and the background imageindicates the distance between the height of the floor surface and theheight of the sensor 2 in the vertical direction.

The measurement unit 14 acquires from the distance image correspondingto the captured image used to identify the circumscribed frame R thedistance z representing a difference between the height of the sensor 2and the height of the feature points P11 or P14 which is the farthestfrom the center of the captured image among the feature points P11, P12,P13, P14 identified in the circumscribed frame R. Herein, the distance zis the information representing the height for each point on the upperface of the conveying object 4 including a real-space pointcorresponding to the feature point P11 or P14. The measurement unit 14calculates an unknown corresponding point P22 in the captured image(which is equivalent to x1 of Expression (5)) by setting x2 ofExpression (5) as the feature point P12 identified in the circumscribedframe R in the captured image, setting h of Expression (5) as the heightof the feature point P12 in the real space, setting z of Expression (5)as the height of the unknown corresponding point P24 on the upper faceconforming to the horizontal-direction position of the feature point P14in the real space (which is equivalent to the height of the featurepoint P11). In addition, the measurement unit 14 calculates an unknowncorresponding point P23 in the captured image (which is equivalent to x1of Expression (5)) by setting x2 of Expression (5) as the feature pointP13 identified in the circumscribed frame R in the captured image,setting h of Expression (5) as the height of the feature point P13 inthe real space, and setting z of Expression (5) as the height of theunknown corresponding point P23 on the upper face conforming to thehorizontal-direction position of the feature point P13 in the real space(which is equivalent to the height of the feature point P11).

The measurement unit 14 identifies corresponding points P22′, P23′, atwhich the line segment defined between the corresponding points P22 andP23 crosses the circumscribed frame R, as corresponding points in thecaptured image corresponding to points on the upper face of theconveying object 4. The measurement unit 14 identifies a rectangularregion defined by the corresponding points P22′, P23′ and the featurepoints P11, P14 representing the upper face as the first specific regionR1 which can be presumed as a rendition of the upper face of theconveying object 4 in the third pattern. The measurement unit 14 setsP11, P22′, P23′, P14 as feature points of the first specific region R1(first corresponding points).

The measurement unit 14 acquires from the storage unit the distanceimage corresponding to the captured image used to identify thecircumscribed frame R of the conveying object 4. The measurement unit 14acquires the height z of the feature point P11 of the first specificregion R1 in the distance image. The measurement unit 14 sets x1 ofExpression (5) as the position in the captured image corresponding tothe feature point P11 specified in the first specific region R1, sets zof Expression (5) as the height of the feature point P11 representingthe upper face of the conveying object 4, and sets h of Expression (5)as the height at which the conveying vehicle 3 comes in contact with theconveying object 4, thus inputting those values into Expression (5).Accordingly, it is possible for the measurement unit 14 to calculate theposition x2 in the captured image in correspondence with thecorresponding point P31 conforming to the horizontal-direction positionof the feature point P11 in the real space.

Similarly, according to Expression (5), the measurement unit 14calculates the corresponding points P32, P33, P34 at the height h′ atwhich the conveying vehicle 3 comes in contact with the conveying object4 in correspondence with the feature points P22′, P23′, P14. Themeasurement unit 14 calculates the region defined by the correspondingpoints P31, P32, P33, P34 (second corresponding points) as the secondspecific region R2 representing the region of the conveying object 4 inthe captured image at the height h′ at which the conveying vehicle 3comes in contact with the conveying object 4. In the case of thecircumscribed frame R having a shape other than a rectangular shape, themeasurement unit 14 may identify a plurality of feature points of thecircumscribed frame R so as to calculate a region defined by a pluralityof corresponding points corresponding to a plurality of feature pointsas the second specific region R2. The measurement unit 14 outputs to thecontact-position identification unit 15 the information of the secondspecific region R2 representing the region of the conveying object 4 inthe captured image at the height h′ at which the conveying vehicle 3comes in contact with the conveying object 4.

The aforementioned process of the measurement unit 14 for the firstspecific region R1 according to the second pattern and the third patternis one manner of identifying a first specific region representing aregion of a target object conforming to the height of an upper faceamong multiple specific regions based on four first corresponding pointsupon calculating four first corresponding points according to therelationship between four feature points of a circumscribed frame R(e.g., a target-object-encompassing region) and the height of fourfeature points as well as the relationship between two or threecorresponding points of a target object, which conform tohorizontal-direction coordinates (or the horizontal-direction position)of feature points of the circumscribed frame R but have differentcoordinates in height, and the height of the upper face.

In addition, the aforementioned process of the measurement unit 14 forthe second specific region R2 according to the second pattern and thethird pattern is one manner of identifying a second specific regionrepresenting a region of a target object conforming to the height of acontact position among multiple specific regions based on four secondcorresponding points upon calculating four second corresponding pointsat the height of a contact position according to the relationshipbetween four first corresponding points of a first specific region andthe height of four first corresponding points as well as therelationship between four second corresponding points, which conform tohorizontal coordinates of first corresponding points at the height ofthe contact position of the target object serving as a secondpredetermined height and whose coordinates in height representcoordinates of the contact position in height, and the height of thecontact position.

Second Exemplary Embodiment

The foregoing exemplary embodiment refers to the measuring system 100configured to determine the contact position and the size of theconveying object 4 subjected to measurement at a predetermined height.However, the measuring system 100 is not necessarily limited to thisusage.

FIG. 16 is a schematic configuration diagram of a control systemequipped with a control device according to the second exemplaryembodiment.

When the conveying object 4 is mounted on a belt conveyer 5 installed inthe control system and moving in a predetermined direction, for example,a control device 1 included in the control system may identify thecontact position with respect to the conveying object 4 so as to controla robot 6 installed in the control system according to the contactposition of the conveying object 4.

In this case, similar to the first exemplary embodiment, the controldevice 1 is configured to identify a circumscribed frame R (e.g., atarget-object-encompassing region) representing a region in which pixelinformation including the conveying object 4 has been changed in theimage information including the conveying object 4 according to anychange between the image information precluding the conveying object 4,which is acquired from the sensor 2, and the image information includingthe conveying object 4. Similar to the first exemplary embodiment, thecontrol device 1 is configured to identify the circumscribed frame R ofthe conveying object 4, the first specific region R1 (e.g., anupper-face region), and the second specific region R2 (e.g., a region ofthe contact position in height), thus identifying the contact positionbased on the second specific region R2. In the case of the secondspecific region R2 having a rectangular shape, for example, it ispossible to identify the contact position of the conveying object 4 withthe robot 6 as the center for each of two sides upon identifying themoving direction of the belt conveyer 5 and two opposite sides having asmall angle with each side of a rectangular shape.

For example, the measuring system 100 may determine the size of packsand the occupation ratio of packs in a pack storage from a region of apack storage for storing packs subjected to measurement.

FIG. 17 is a schematic illustration showing an example of the placementrelationship of packs serving as target objects subjected to measurementin a pack storage according to another exemplary embodiment.

The measurement system 100 is equipped with the sensor 2 configured toproduce captured images of packs 162 in a pack storage 161 in adirection from upwards to downwards. Similar to the aforementioned otherexemplary embodiments, the control device 1 installed in the measuringsystem 100 may determine the circumscribed frame R for each packdetected based on its captured image as well as the first specificregion R1 (i.e., an upper-face region) and the second specific region R2(i.e., a region at a predetermined height). The control device 1 maycalculate the second specific region R2 of the pack 162 detected basedon its captured image and the space of the pack storage 161 (e.g., avacant space 163 and a path space 164) in the captured image. In thiscase, the control device 1 should store a region of a pack storage inadvance so as to determine a region failing to detect the secondspecific region R2 at the predetermined height of the pack 162. In thisconnection, the control device 1 may determine the region failing todetect the second specific region R2 as the vacant space or the pathspace 164.

In addition, the measuring system 100 may determine the size of a targetobject and the size of a path allowing for the target object to passtherethrough so as to determine whether to allow or disallow passage, orit may extract a passable path to determine a conveying path towards aplace of conveyance. The measuring system 100 may allow a worker or asite-supervisor to remotely grasp the size of a target object or todetermine the size of a pack moving along a belt conveyor.

Third Exemplary Embodiment

FIG. 18 is a schematic configuration diagram of a measuring systemaccording to the third exemplary embodiment.

With reference to FIG. 18 , the measuring system 100 includes thecontrol device 1, the sensor 2, and the conveying object 4. The controldevice 1 can communicate with the sensor 2 through networks.

FIG. 18 is a schematic configuration diagram of the control device 1according to the third exemplary embodiment, for example, wherein thecontrol device 1 may be an information processing device such as acomputer.

FIG. 19 is a functional block diagram of the control device 1 accordingto the third exemplary embodiment.

The control device 1 can be embodied by cloud computing. The controldevice 1 includes the difference detection unit 13 and the measurementunit 14.

The difference detection unit 13 is configured to detect a regioncausing any change between the first image information and the secondimage information based on the first image information not including atarget object in an imageable area and the second image informationincluding the target object in the imageable area.

The measurement unit 14 is configured to determine atarget-object-encompassing region representing the region causingchanges and to thereby determine the two-dimensional size of a targetobject included in the target-object-encompassing region based on theheight information and the target-object-encompassing region.

FIG. 20 is a flowchart showing a flow of processes implemented by thecontrol device according to the third exemplary embodiment.

The control device 1 which can communicate with a sensor configured toacquire the height information of a target object and the imageinformation of the target object includes at least the differencedetection unit 13 and the measurement unit 14.

The difference detection unit 13 detects a region causing any change inthe pixel information including a target object in the image informationof the target object according to changes between the image informationnot including the target object and the image information including thetarget object (step S181).

The measurement unit 14 determines the two-dimensional size of a targetobject included in the target-object-encompassing region based on thepredetermined height information and the target-object-encompassingregion representing the region causing changes in the image information(step S182).

FIG. 21 is a schematic diagram showing an outline of processing of thecontrol device according to the third exemplary embodiment.

To determine the two-dimensional size of an upper face of the conveyingobject 4, the measurement unit 14 detects the relationship betweenfeature points (P12, P13, P14) and the height information of featurepoints (P12, P13, P14) in a target-object-encompassing region (R)including all regions of target objects reflected in the imageinformation. In this connection, the height information of the featurepoints P12, P13, P14 is obvious from the distance information outputfrom the sensor 2. In addition, the measurement unit 14 detects therelationship between the height information representing thepredetermined height (or the upper-face height) and the correspondingpoints (P22, P23, P24) conforming to the horizontal-direction positions(or the horizontal-direction coordinates) of the feature points (P12,P13, P14) and located at the predetermined height (or the upper-faceheight). In this connection, the upper-face height information isobvious from the distance information of the feature point P11 outputfrom the sensor 2. According to the aforementioned relationship, themeasurement unit 14 calculates the corresponding points (P11, P22, P23,P24) at the upper-face height. Accordingly, the measurement unit 14determines the region covering the corresponding points P11, P22, P23,P24 as a specific region (or a first specific region) representing theupper-face size. The first specific region is an example a regionrepresenting the two-dimensional size of a target object.

To determine the two-dimensional size of the conveying object 4 at thecontact position, the measurement unit 14 detects the relationshipbetween the feature points (P11, P12, P13, P14) in the specific region(or the first specific region) representing the upper-face size and theheight information of the feature points (P11, P12, P13, P14). In thisconnection, the height information of the feature points P11, P12, P13,P14 is obvious from the distance information output from the sensor 2.In addition, the measurement unit 14 detects the relationship betweenthe height information representing the predetermined height (or theheight of the contact position) and the corresponding points (P31, P32,P33, P34) conforming to the horizontal-direction positions (or thehorizontal-direction coordinates) of the feature points (P11, P22, P23,P24) and located at the predetermined height (e.g., the height of thecontact position at which a conveying vehicle may come in contact with aconveying object). The height information of the contact position hasbeen prescribed in advance. According to the aforementionedrelationship, the measurement unit 14 calculates the correspondingpoints (P31, P32, P33, P34) at the height of the contact position. Themeasurement unit 14 determines the region covering the correspondingpoints P31, P32, P33, P34 as a specific region (or a second specificregion) representing the upper-face size. The second specific region isan example of a region representing the two-dimensional size of a targetobject.

The aforementioned control device 1 includes a computer system therein.The aforementioned processes are stored on computer-readable storagemedia in the form of programs, wherein a computer may read and executeprograms to achieve the aforementioned processes. Herein, thecomputer-readable storage media refer to magnetic disks, magneto-opticaldisks, CD-ROM, DVD-ROM, semiconductor memory and the like. In addition,it is possible to deliver computer programs to a computer throughcommunication lines, and therefore the computer receiving programsdelivered thereto may execute programs.

The aforementioned programs may achieve some of the foregoing functions.Alternatively, the aforementioned programs may be so-called differentialfiles (or differential programs) which can achieve the foregoingfunctions when combined with pre-recorded programs of the computersystem.

REFERENCE SIGNS LIST

-   1 . . . control device (measuring apparatus)-   2 . . . sensor-   3 . . . conveying vehicle-   4 . . . conveying object (target object)-   5 . . . belt conveyer-   6 . . . robot-   11 . . . image-information acquisition unit-   12 . . . distance-information acquisition unit-   13 . . . difference detection unit-   14 . . . measurement unit-   15 . . . contact-position identification unit-   16 . . . conveyance control unit-   17 . . . display unit-   100 . . . measuring system

What is claimed is:
 1. A measuring method comprising: determining atarget-object-encompassing region representing a region causing a changebetween first image information and second image information based onthe first image information precluding a target object in an imageablearea and the second image information including the target object in theimageable area; and determining a two-dimensional size of the targetobject included in the target-object-encompassing region based on thetarget-object-encompassing region and height information.
 2. Themeasuring method according to claim 1, wherein the two-dimensional sizeof the target object included in the target-object-encompassing regionat a predetermined height is determined based on thetarget-object-encompassing region and the height information.
 3. Themeasuring method according to claim 2, comprising: calculating acorresponding point used to determine the two-dimensional size of thetarget object at the predetermined height according to a relationshipbetween a feature point of the target-object-encompassing region andheight information of the feature point as well as a relationshipbetween the corresponding point, which conforms to ahorizontal-direction position of the feature point and is located at thepredetermined height, and the height information representing thepredetermined height, thus determining a specific region representingthe two-dimensional size in the second image information based on thecorresponding point.
 4. The measuring method according to claim 3,comprising: calculating a plurality of corresponding points at thepredetermined height according to a relationship between a plurality offeature points of the target-object-encompassing region and heightinformation of the plurality of feature points as well as a relationshipbetween the plurality of corresponding points, which conform tohorizontal-direction positions of the plurality of feature points andare located at the predetermined height, and the height informationrepresenting the predetermined height, thus determining the specificregion serving as one aspect of the two-dimensional size based on theplurality of corresponding points.
 5. The measuring method according toclaim 4, comprising: calculating a plurality of first correspondingpoints at the upper-face height of the target object according to arelationship between a plurality of feature points of thetarget-object-encompassing region and height information of theplurality of feature points as well as a relationship between theplurality of first corresponding points, which conform tohorizontal-direction positions of the plurality of feature points at theupper-face height of the target object serving as a first predeterminedheight, and the height information representing the upper-face height ofthe target object, thus determining a first specific region representinga region of the target object corresponding to the upper-face height ofthe specific region based on the plurality of first correspondingpoints; and calculating a plurality of second corresponding points at aheight of a contact position according to a relationship between theplurality of first corresponding points of the first specific region andthe height information of the plurality of first corresponding points aswell as a relationship between the plurality of second correspondingpoints, which conform to horizontal-direction positions of the pluralityof first corresponding points and are located at the height of thecontact position serving as a second predetermined position, and heightinformation representing the height of the contact position, thusdetermining a second specific region representing a region of the targetobject corresponding to the height of the contact position in thespecific region based on the plurality of second corresponding points.6. The measuring method according to claim 3, wherein the contactposition of the target object is determined based on the specificregion.
 7. A measuring apparatus comprising a processor and a memoryconfigured to store instructions, the processor executing theinstructions to: determine a target-object-encompassing regionrepresenting a region causing a change between first image informationand second image information based on the first image informationprecluding a target object in an imageable area and the second imageinformation including the target object in the imageable area; anddetermine a two-dimensional size of the target object included in thetarget-object-encompassing region based on thetarget-object-encompassing region and height information.
 8. Themeasuring apparatus according to claim 7, wherein the processor isconfigured to determine the two-dimensional size of the target objectincluded in the target-object-encompassing region at a predeterminedheight based on the target-object-encompassing region and the heightinformation.
 9. The measuring apparatus according to claim 8, whereinthe processor is configured to calculate a corresponding point used todetermine the two-dimensional size of the target object at thepredetermined height according to a relationship between a feature pointof the target-object-encompassing region and height information of thefeature point as well as a relationship between the corresponding point,which conforms to a horizontal-direction position of the feature pointand is located at the predetermined height, and the height informationrepresenting the predetermined height, thus determining a specificregion representing the two-dimensional size in the second imageinformation based on the corresponding point.
 10. The measuringapparatus according to claim 9, wherein the processor is configured tocalculate a plurality of corresponding points at the predeterminedheight according to a relationship between a plurality of feature pointsof the target-object-encompassing region and height information of theplurality of feature points as well as a relationship between theplurality of corresponding points, which conform to horizontal-directionpositions of the plurality of feature points, and the height informationrepresenting the predetermined height, thus determining the specificregion serving as one aspect of the two-dimensional size based on theplurality of corresponding points.
 11. The measuring apparatus accordingto claim 10, wherein the processor is configured to calculate aplurality of first corresponding points at the upper-face height of thetarget object according to a relationship between a plurality of featurepoints of the target-object-encompassing region and height informationof the plurality of feature points as well as a relationship between theplurality of first corresponding points, which conform tohorizontal-direction positions of the plurality of feature points at theupper-face height of the target object serving as a first predeterminedheight, and the height information representing the upper-face height ofthe target object, thus determining a first specific region representinga region of the target object corresponding to the upper-face height ofthe specific region based on the plurality of first correspondingpoints, and wherein the processor is configured to calculate a pluralityof second corresponding points at a height of a contact positionaccording to a relationship between the plurality of first correspondingpoints of the first specific region and the height information of theplurality of first corresponding points as well as a relationshipbetween the plurality of second corresponding points, which conform tohorizontal-direction positions of the plurality of first correspondingpoints and are located at the height of the contact position serving asa second predetermined position, and height information representing theheight of the contact position, thus determining a second specificregion representing a region of the target object corresponding to theheight of the contact position in the specific region based on theplurality of second corresponding points.
 12. The measuring apparatusaccording to claim 9, wherein the contact position of the target objectis determined based on the specific region.
 13. A measuring systemcomprising: determining a target-object-encompassing region representinga region causing a change between first image information and secondimage information based on the first image information precluding atarget object in an imageable area and the second image informationincluding the target object in the imageable area; and determining atwo-dimensional size of the target object included in thetarget-object-encompassing region based on thetarget-object-encompassing region and height information.
 14. Themeasuring system according to claim 13, wherein the two-dimensional sizeof the target object included in the target-object-encompassing regionat a predetermined height is determined based on thetarget-object-encompassing region and the height information.
 15. Themeasuring system according to claim 14, comprising: calculating acorresponding point used to determine the two-dimensional size of thetarget object at the predetermined height according to a relationshipbetween a feature point of the target-object-encompassing region andheight information of the feature point as well as a relationshipbetween the corresponding point, which conforms to ahorizontal-direction position of the feature point and is located at thepredetermined height, and the height information representing thepredetermined height, thus determining a specific region representingthe two-dimensional size in the second image information based on thecorresponding point.
 16. The measuring system according to claim 15,comprising: calculating a plurality of corresponding points at thepredetermined height according to a relationship between a plurality offeature points of the target-object-encompassing region and heightinformation of the plurality of feature points as well as a relationshipbetween the plurality of corresponding points, which conform tohorizontal-direction positions of the plurality of feature points andare located at the predetermined height, and the height informationrepresenting the predetermined height, thus determining the specificregion serving as one aspect of the two-dimensional size based on theplurality of corresponding points.
 17. The measuring system according toclaim 16, comprising: calculating a plurality of first correspondingpoints at the upper-face height of the target object according to arelationship between a plurality of feature points of thetarget-object-encompassing region and height information of theplurality of feature points as well as a relationship between theplurality of first corresponding points, which conform tohorizontal-direction positions of the plurality of feature points at theupper-face height of the target object serving as a first predeterminedheight, and the height information representing the upper-face height ofthe target object, thus determining a first specific region representinga region of the target object corresponding to the upper-face height ofthe specific region based on the plurality of first correspondingpoints; and calculating a plurality of second corresponding points at aheight of a contact position according to a relationship between theplurality of first corresponding points of the first specific region andthe height information of the plurality of first corresponding points aswell as a relationship between the plurality of second correspondingpoints, which conform to horizontal-direction positions of the pluralityof first corresponding points and are located at the height of thecontact position serving as a second predetermined position, and heightinformation representing the height of the contact position, thusdetermining a second specific region representing a region of the targetobject corresponding to the height of the contact position in thespecific region based on the plurality of second corresponding points.18. The measuring system according to claim 15, wherein the contactposition of the target object is determined based on the specificregion.